Display method and display apparatus in which a part of a screen area is in a through-state

ABSTRACT

Display control that extends a user&#39;s visual ability is accomplished. By wearing an eyeglass-type or headgear-type mounting unit, he or she is allowed to see display means disposed in front of his or her eyes. By causing a part of a screen area of the display means to be in a through-state and a display with a display image signal to be executed, the user can see an image of a scene different from a scene that he or she ordinarily sees with a display of the display image signal while he or she can see the ordinary visual scene with the through-state area.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims the benefit of priorityfrom allowed U.S. Ser. No. 12/445,477, filed Sep. 21, 2009, the entirecontents of which is incorporated herein by reference. U.S. Ser. No.12/445,477 is a National Stage Application of PCT/JP2007/070397, filedOct. 12, 2007, which claims the benefit of priority from Japanese PatentApplication No. 2006-281042, filed Oct. 16, 2006.

TECHNICAL FIELD

The present invention relates to a display apparatus and a displaymethod that display an image in front of the eyes of the user who iswearing, for example, an eyeglass-type or headgear-type mounting unit.

BACKGROUND ART

Various types of apparatus that use an eyeglass-type or headgear-typemounting unit having display sections that are disposed in front of theeyes of the user and display an image have been proposed, for example,in Japanese Patent Application Laid-Open Publication No. HEI 8-126031,Japanese Patent Application Laid-Open Publication No. HEI 9-27970, andJapanese Patent Application Laid-Open Publication No. HEI 9-185009.

However, apparatus that control image capturing and displaying from viewpoints of aiding of user's visual sense and improvement of his or hervisual ability have not been known.

DISCLOSURE OF THE INVENTION

Therefore, an object of the present invention is to accomplish aidingand improvement of user's visual sense.

A display apparatus of the present invention includes display means,disposed in front of eyes of a user, for displaying an image and causinga screen area on which an image is displayed to be in a through-statewhich is transparent or semi-transparent; image signal generation meansfor generating a display image signal of a scene different from a scenewhich the user sees through the display means when the display means iscaused to be in the through-state; and control means for controlling apart of the screen area of the display means to be in the through-stateand the display means to execute displaying with the display imagesignal generated by the image signal generation means.

The image signal generation means includes an image capturing section;and a signal processing section which performs a signal process for acaptured image signal obtained by the image capturing section.

In this case, the image capturing section is an image capturing sectiondisposed such that the image capturing section performs capturing in adirection which the user sees through the display means when the screenarea is caused to be in the through-state as a direction of an object.

The image capturing section can be an image capturing section disposedsuch that the image capturing section performs capturing in a directiondifferent from a direction which the user sees through the display meanswhen the screen area is caused to be in the through-state as a directionof an object.

Instead, the image capturing section is an image capturing sectioncapable of changing a direction of an object.

Instead, the image signal generation means includes a reception sectionwhich receives an image signal from an external device.

Instead, the image signal generation means includes a signal processingsection which performs a signal process for a supplied image signal.

Instead, the control means sets a parent screen area and a child screenarea in the screen area of the display means and causes one of theparent screen area and the child screen area to be in the through-stateand the other to be in a screen display state with the display imagesignal.

Instead, the control means divides the screen area of the display meansinto areas and causes one of the divided areas to be in thethrough-state and the other to be in an image display state with thedisplay image signal.

Instead, the display image signal is an image signal obtained bycapturing of an image of a near scene or an image of a far scene.

Instead, the display image signal is an image signal obtained bycapturing of a telescopic image or a wide-angle image.

Instead, the display image signal is an image signal obtained by anenlargement process or a reduction process.

Instead, the display image signal is an image signal obtained bycapturing whose image capturing sensitivity is increased or decreased.

Instead, the display image signal is an image signal obtained bycapturing whose infrared image capturing sensitivity is increased.

Instead, the display image signal is an image signal obtained bycapturing whose ultraviolet image capturing sensitivity is increased.

Instead, the display image signal is an image signal obtained bycapturing in a direction different from a direction which the user seesthrough the display means when the screen area is caused to be in thethrough-state as a direction of an object.

Instead, the image capturing section controls an operation of an imagecapturing lens system such that the image signal generation meansobtains a display image signal as a scene different from a scene whichthe user sees when the display means is caused to be in thethrough-state.

Instead, the signal processing section performs a signal process suchthat the image signal generation means obtains a display image signal asa scene different from a scene which the user sees when the displaymeans is caused to be in the through-state.

Instead, the receiving section receives an image signal captured by anexternal image capturing device such that the image signal generationmeans obtains the display image signal as a scene different from a scenewhich the user sees when the display means is caused to be in thethrough-state.

Instead, the display apparatus further includes external informationobtaining means for obtaining external information. The control meanscauses the display image signal to be displayed on the display meansbased on the external information obtained from the external informationobtaining means.

Instead, the control means controls an area on which the display imagesignal is displayed based on the external information obtained from theexternal information obtaining means.

A display method of the present invention is a display method of adisplay apparatus having display means, disposed in front of eyes of auser, for displaying an image and causing a screen area on which animage is displayed to be in a through-state which is transparent orsemi-transparent, including a step of generating a display image signalof a scene different from a scene which the user sees through thedisplay means when the display means is caused to be in thethrough-state; and a step of controlling a part of the screen area ofthe display means to be in the through-state and the display means toexecute displaying with the generated display image signal.

According to the present invention, when the user wears the displayapparatus, for example, as an eyeglass-type or head-gear type mountingunit, he or she is caused to see display means disposed in front of hisor her eyes. The display means can be caused to be in the through-state.

By causing a part of the screen of the display means to be in thethrough-state and a display image signal generated by image signalgeneration means to be displayed, while being caused to see an ordinaryvisual scene in the through-state area, the user can see an image of ascene different from the ordinary visual scene with the display of thedisplay image signal. For example, while seeing ahead with thethrough-state area, the user can see a telescopic image, an enlargedimage, a specially captured image, an image of a rear scene, and soforth with a display of the display image signal.

In other words, since the user is caused to see a scene that he or shecannot see with his or her ordinary visual sense together with his orher ordinary visual scene, the visual ability is caused to be extended.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A and FIG. 1B are explanatory diagrams of exemplary appearances ofa display apparatus of an embodiment of the present invention;

FIG. 2A and FIG. 2B are explanatory diagrams of other exemplaryappearances of a display apparatus of an embodiment;

FIG. 3A, FIG. 3B, and FIG. 3C are explanatory diagrams of therelationship between a display apparatus of the embodiment and externaldevices;

FIG. 4 is a block diagram showing a display apparatus of an embodiment;

FIG. 5 is another block diagram of a display apparatus of an embodiment;

FIG. 6 is a still another block diagram of a display apparatus of anembodiment;

FIG. 7A, FIG. 7B, FIG. 7C, FIG. 7D, FIG. 7E, and FIG. 7F are explanatorydiagrams of areas of display sections of an embodiment;

FIG. 8A, FIG. 8B, FIG. 8C, and FIG. 8D are explanatory diagrams of areasof display sections of an embodiment;

FIG. 9 is a flow chart of a control process of an embodiment;

FIG. 10A and FIG. 10B are explanatory diagrams of image display statesof an embodiment;

FIG. 11A and FIG. 11B are explanatory diagrams of image display statesof an embodiment;

FIG. 12A and FIG. 12B are explanatory diagrams of image display statesof an embodiment;

FIG. 13A and FIG. 13B are explanatory diagrams of image display statesof an embodiment;

FIG. 14A and FIG. 14B are explanatory diagrams of image display statesof an embodiment; and

FIG. 15A, FIG. 15B, FIG. 15C, and FIG. 15D are explanatory diagrams ofimage display states of an embodiment.

BEST MODES FOR CARRYING OUT THE INVENTION

Next, the display apparatus and display method according to the presentinvention will be described in the following order.

[1. Exemplary appearance of display apparatus and relationship withexternal device][2. Exemplary structure of display apparatus][3. Areas on screen][4. Exemplary display operation][5. Exemplary operation triggers][6. Effect of embodiment, exemplary modification, and exemplaryextension]

1. Exemplary Appearance of Display Apparatus and Relationship withExternal Device

As an embodiment, FIG. 1A and FIG. 1B show exemplary appearances of adisplay apparatus 1 that is an eyeglass-type display. The displayapparatus 1 has a mounting unit that is formed in a half-circumferentialframe that fits bilateral temporal regions and a occipital region of theuser, for example, as shown in FIG. 1B and the display apparatus 1 isworn by the user in such a manner that the mounting unit is hung on bothhis or her conchae as shown in FIG. 1A.

When the display apparatus 1 is worn by the user, a pair of displaysections 2, 2 for the left and right eyes are disposed immediately infront of both his or her eyes, namely at the lens positions of ordinaryeyeglasses. The respective display sections 2 are, for example, composedof liquid crystal panels and by controlling their transmittances thedisplay sections 2 can be caused to be in the through-state, namelytransparent or semitransparent state as shown in the drawing. When thedisplay sections 2 are caused to be in the through-state, even if theuser wears the display apparatus 1 all times like eyeglasses, it doesnot affect his or her ordinary life.

While the user is wearing the display apparatus 1, an image capturinglens 3 a is disposed in the forward direction of the user. In this case,the image capturing lens 3 a is caused to capture an image in thedirection that the user sees through the display sections 2 as thedirection of the object when they are caused to be in the through-state.

In addition, a light emission section 4 a that lights up the imagecapturing direction of the image capturing lens 3 a is disposed. Thelight emission section 4 a is composed, for example, of an LED (LightEmitting Diode).

Since FIG. 1A and FIG. 1B are just exemplary, it can be contemplatedthat there are many structures in which the user wears the displayapparatus 1, generally as long as the display apparatus 1 is aneyeglass-type or headgear-type mounting unit and at least according tothis embodiment, the display sections 2 are disposed immediately infront of the eyes of the user. In addition, as well as the structure inwhich the display sections 2 are disposed corresponding to both the eyesof the user, one display section 2 may be disposed for one eye of theuser.

It can be contemplated that the light emission section 4 a is notprovided.

In FIG. 1A, although the image capturing lens 3 a is disposed such thatthe direction of the object is the forward direction of the user, theimage capturing lens 3 a may be disposed such that when the displaysections 2 are caused to be in the through-state, the direction of theobject is different from the direction that user sees through thedisplay sections 2.

FIG. 2A and FIG. 2B show an example. In the example shown in FIG. 2A andFIG. 2B, the image capturing lens 3 a is not disposed in the forwarddirection, but the image capturing lens 3 a and the light emissionsection 4 a are mounted on a unit disposed on the rear head side. Inother words, in this case, the image capturing lens 3 a is caused tocapture an image in the backward direction that the user cannotordinarily see.

Although not shown, the image capturing lens 3 a may be disposed suchthat the direction of the object is the upward direction, left sidedirection, right side direction, downward direction, or the like of theuser.

In the examples shown in FIG. 1A and FIG. 2B, since the image capturinglens 3 a is fixedly mounted, the direction of the object is fixed whenan image thereof is captured (in the forward direction or backwarddirection of the user). However, when the image capturing lens 3 a ismounted through a moving mechanism that can change the direction of theobject, it can be changed manually or automatically when an image iscaptured.

In FIG. 1A and FIG. 2B, although one image capturing functional block isdisposed as the image capturing lens 3 a, a plurality of image capturinglenses 3 a may be mounted so as to provide a plurality of imagecapturing functional blocks.

An image signal captured by an image capturing functional system,including the image capturing lens 3 a, which will be described later,is processed in a predetermined manner and the resultant display imagesignal is displayed on the display sections 2 through a predeterminedprocess.

In this case, a partial area of the screen of each of the displaysections 2 is caused to be in the through-state and an image based onthe display image signal is displayed on another area.

Although an exemplary internal structure will be described later, it canbe contemplated that the display apparatus 1 has a communicationfunction that communicates with an external device (a communicationsection 26 shown in FIG. 5).

Thus, the sources of the display image signals displayed on the displaysections 2 are supposed to be not only the image capturing functionalblock including the image capturing lens 3 a but also the communicationfunctional section. In other words, an image signal transmitted fromanother image capturing device or the like as an external device can bereceived by the communication functional section and displayed on thedisplay sections 2.

FIG. 3A, FIG. 3B, and FIG. 3C show exemplary modes of usage of thedisplay apparatus 1 in association with external devices.

FIG. 3A shows the case in which the display apparatus 1 is used alone.In this case, since the display apparatus 1 has an image capturingfunction, a display image signal generated from a captured image signalas a source can be displayed on the display sections 2.

FIG. 3B shows an example in which the display apparatus 1 has acommunication function and communicates with an external image capturingdevice 70. In this case, the display apparatus 1 receives an imagecaptured by the image capturing device 70 and causes the displaysections 2 to display the image. A video camera, a digital camera, orthe like that has a communication function can be supposed to be theexternal image capturing device 70 and instead, as shown in FIG. 1A andFIG. 1B, the display apparatus 1 having an image capturing function canbe contemplated to be an external image capturing device 70 for thedisplay apparatus 1.

In addition, the external image capturing device 70 can be contemplatedto be various types such as an image capturing device that belongs tothe user of the display apparatus 1, an image capturing device thatbelongs to an acquaintance of the user of the display apparatus 1, andan image capturing device that belongs to a public or private companythat provides an image and that can communicate with the displayapparatus 1.

FIG. 3C shows an example in which the display apparatus 1 is providedwith a communication function, in particular, a communication accessfunction through a network 73 such as the Internet so as to communicatewith the external image capturing device 70 connected through thenetwork 73. In this case, the display apparatus 1 receives a capturedimage signal through the network 73 and causes the display sections 2 todisplay an image with a display image signal based on the receivedcaptured image signal.

2. Exemplary Structure of Display Apparatus

FIG. 4 shows an exemplary internal structure of a display apparatus 1.

A system controller 10 is composed of a microcomputer that includes, forexample, a CPU (Central Processing Unit), a ROM (Read Only Memory), aRAM (Random Access Memory), an nonvolatile memory section, and aninterface section and is a control section that controls all thesections of the display apparatus 1.

The system controller 10 controls each section of the display apparatus1 based on an internal operation program and an operation triggerreceived from the operation trigger information generation section 16and causes the display sections 2 to display a predetermined image.

Disposed in the display apparatus 1 are an image capturing section 3, animage capturing control section 11, and a captured image signalprocessing section 15.

The image capturing section 3 includes a lens system having the imagecapturing lens 3 a shown in FIG. 1A or FIG. 2B, a diaphragm, a zoomlens, a focus lens, and so forth, a driving system that causes the lenssystem to perform a focus operation and a zoom operation, a solid stateimage sensor array that detects light of a captured image obtained bythe lens system, converts light into electricity, and generates acaptured image signal corresponding to the electricity, and so forth.The solid state image sensor array is composed, for example, of a CCD(Charge Coupled Device) sensor array or a CMOS (Complementary MetalOxide Semiconductor) sensor array.

In the case of the example of FIG. 1A and FIG. 1B, the image capturingsection 3 captures an image of a scene in the forward direction of theuser, whereas in the case of the example of FIG. 2A and FIG. 2B, theimage capturing section 3 captures an image of a scene in his or herbackward direction.

The captured image signal processing section 15 includes a samplehold/AGC (Automatic Gain Control) circuit that adjusts the gain of andtrims the waveform of the signal obtained by the solid state imagesensor array of the image capturing section 3 and a video A/D converterand obtains a captured image signal as digital data. The captured imagesignal processing section 15 performs a white balance process, aluminance process, a color signal process, a vibration correctionprocess, and so forth for the captured image signal.

The image capturing control section 11 controls the operations of theimage capturing section 3 and the captured image signal processingsection 15 based on commands received from the system controller 10. Theimage capturing control section 11 controls, for example, turning-on andoff the operations of the image capturing section 3 and the capturedimage signal processing section 15. In addition, the image capturingcontrol section 11 is contemplated to control (control of motors of) theimage capturing section 3 to perform an auto focus operation, anautomatic exposure adjustment operation, an aperture adjustmentoperation, a zoom operation, and so forth.

When the moving mechanism that can change the direction of the object ofthe image capturing lens 3 a is provided, the image capturing controlsection 11 controls the moving mechanism to change the direction of theimage capturing lens 3 a of the image capturing section 3 under thecontrol of the system controller 10.

In addition, the image capturing control section 11 includes a timinggenerator and controls signal process operations of the sample hold/AGCcircuit and the video A/D converter of the solid state image sensorarray and the image capturing control section 11 with a timing signalgenerated by the timing generator. In addition, the image capturingcontrol section 11 can perform the timing control to change the framerate of a capturing image.

In addition, the image capturing control section 11 controls the imagecapturing sensitivity and the signal process of the solid state imagesensor array and the captured image signal processing section 15. Tocontrol an image capturing sensitivity, the image capturing controlsection 11 controls, for example, the gain of the signal that has beenread from the solid state image sensor array, black level setting,various types of coefficients of digital data of the captured imagesignal process, a correction amount of the vibration correction process,and so forth. With respect to image capturing sensitivity, the imagecapturing control section 11 can perform an overall sensitivityadjustment regardless of a wavelength band and specific sensitivityadjustments for specific wavelength bands (for example, image capturingby eliminating a specific wavelength band) such as an infrared regionand an ultraviolet region. The wavelength specific sensitivityadjustment can be performed by inserting a wavelength filter into theimage capturing lens system or by performing a wavelength filtercalculation process for the captured image signal. In these cases, theimage capturing control section 11 can control the image capturingsensitivity, for example, by inserting a wavelength filter and/ordesignating a filter calculation coefficient.

As a structure that displays data to the user, the display apparatus 1includes the display sections 2, a display image processing section 12,a display driving section 13, and a display control section 14.

A captured image signal that has been captured by the image capturingsection 3 and then processed by the captured image signal processingsection 15 is supplied to the display image processing section 12. Thedisplay image processing section 12 is, for example, a so-called videoprocessor and executes various types of display processes for thesupplied captured image signal.

The display image processing section 12 can perform, for example, aluminance level adjustment, a color correction, a contrast adjustment, asharpness (edge enhancement) adjustment, and so forth for the capturedimage signal. In addition, the display image processing section 12 can,for example, generate an enlarged image of which a part of a capturedimage signal is enlarged or a reduced image, perform image effectprocesses such as soft focus, mosaic, luminance inversion, partial imagehighlight display (emphasized display), change of entire coloratmosphere, separate a captured image for a multiply separated display,combine a plurality of captured images, perform a process of generatinga character image and a graphic image, and superimpose a generated imagewith a captured image. In other words, the display image processingsection 12 can perform various types of processes for a digital imagesignal as a captured image signal.

The display driving section 13 is composed of a pixel driving circuitthat displays an image signal supplied from the display image processingsection 12 on the display sections 2, which are for example liquidcrystal displays. In other words, the display driving section 13 appliesa driving signal based on the image signal to each pixel formed in amatrix shape in the display sections 2 at predeterminedhorizontal/vertical drive timings to cause the display sections 2 todisplay an image. In addition, the display driving section 13 controlsthe transmittance of each pixel of the display sections 2 to cause theentire surface and a part thereof to be in the through-state.

The display control section 14 controls the process and operation of thedisplay image processing section 12 and the operation of the displaydriving section 13 corresponding to commands received from the systemcontroller 10. In other words, the display control section 14 causes thedisplay image processing section 12 to execute the foregoing varioustypes of processes. In addition, the display control section 14 controlsthe display driving section 13 to cause the display sections 2 to switchthe display state between the through-state and the image display state.

In addition, the lighting section 4 and a lighting control section 18are provided. The lighting section 4 is composed of the light emissionsection 4 a (for example, an LED) shown in FIG. 1A and FIG. 2B and alight emission circuit that causes the light emission section 4 a toemit light. The lighting control section 18 causes the lighting section4 to execute a light emission operation corresponding to a commandsupplied from the system controller 10.

Since the light emission section 4 a of the lighting section 4 has beenmounted as shown in FIG. 1A or FIG. 2B, the lighting section 4 performsa lighting operation in the direction of the object of the imagecapturing lens 3 a.

The operation trigger information generation section 16 generatesoperation triggers that cause the display sections 2, for example, tostart/stop displaying of an image and switch the display modes.

The operation trigger information generation section 16 can be composedof operation switches as operation keys or operation dials that the usercontrols and an operation detection mechanism that detects theoperations of the operation switches. In other words, user's manualoperations are operation trigger information of various types ofoperations. The operation trigger information generation section 16supplies user's operation information as operation trigger informationto the system controller 10 to control a display operation correspondingto the user's operation.

In addition, the operation trigger information generation section 16 maybe structured such that it can detect user's information (detectedinformation of user's visual situation, body action, vital situation,and so forth) and external information (detected information ofenvironmental situation of the display apparatus, place, date/time,situation of object, and so forth). For example, the system controller10 may determine whether or not an operation trigger has occurred basedon these user's information and external information. Examples of theuser's information and external information will be described later.

FIG. 5 shows another exemplary structure of the display apparatus 1.Blocks having the same functions as those shown in FIG. 4 are denoted bythe same reference numerals and their redundant description will beomitted.

The structure shown in FIG. 5 is not provided with the image capturingfunctional block (image capturing section 3, image capturing controlsection 11, and captured image signal processing section 15) and thelighting section 4, and the lighting control section 18, but with acommunication section 26.

The communication section 26 transmits and receives data to and from anexternal device. The external device can be contemplated to be one ofvarious types of devices including the image capturing device 70described with reference to FIG. 3B and FIG. 3C.

The communication section 26 may be structured such that it performs anetwork communication through a near range radio communication, forexample, to a network access point according to radio LAN (Local AreaNetwork) system, Bluetooth (registered trademark) system, or the like orperforms a direct radio communication with an external device having acorresponding communication function.

In the example shown in FIG. 5, the communication section 26communicates with an external image capturing device 70 as describedwith reference to FIG. 3B or FIG. 3C and receives a captured imagesignal therefrom. The communication section 26 supplies the receivedcaptured image signal to a display image processing section 12. An imagesignal processed in the display image processing section 12 is suppliedto a display driving section 13 and displayed on the display sections 2.

FIG. 6 shows another exemplary structure of the display apparatus 1. Thesimilar blocks to those shown in FIG. 4 are denoted by the similarreference numerals and their redundant description will be omitted.

The structure shown in FIG. 6 has two image capturing functional blocks(each of which includes image capturing section 3, image capturingcontrol section 11, and captured image signal processing section 15) inthe structure shown in FIG. 4.

In other words, a first image capturing functional block composed of animage capturing section 3X, an image capturing control section 11X, anda captured image signal processing section 15X and a second imagecapturing functional block composed of an image capturing section 3Y, animage capturing control section 11Y, and a captured image signalprocessing section 15Y are provided.

The image capturing sections 3X and 3Y may capture images in the samedirection of an object or images in different directions of objects.

For example, the image capturing sections 3X and 3Y may be disposed suchthat they capture images in the forward direction. Instead, the imagecapturing sections 3X and 3Y may be disposed such that the imagecapturing section 3X captures an image in the forward direction and theimage capturing section 3Y captures an image in the backward direction.Instead, the image capturing sections 3X and 3Y may be disposed suchthat the image capturing section 3X captures an image in the rightbackward direction and the image capturing section 3Y captures an imagein the left backward direction. Instead, the image capturing sections 3Xand 3Y may be disposed such that the image capturing section 3X capturesan image in the downward direction and the image capturing section 3Ycaptures an image in the upward direction.

Captured image signals obtained by the individual image capturingfunctional blocks are processed by a display image processing section12, supplied to a display driving section 13, and then displayed ondisplay sections 2.

Although exemplary structures of the display apparatus 1 have been shownin FIG. 4, FIG. 5, and FIG. 6, the display apparatus 1 may be supposedto have more various structures.

For example, it can be contemplated that three or more image capturingfunctional blocks are provided.

A communication section 26 that receives a captured image signal from anexternal image capturing device 70 can be contemplated to be providedalong with one or a plurality of image capturing functional blocks.

In addition, the image capturing section 3 (image capturing lens 3 a) inthe image capturing functional block may be contemplated to be fixed inthe direction of an object or movable therefrom. However, when aplurality of image capturing functional blocks are provided, it can becontemplated that all of them are fixed or movable or a part of them arefixed and the rest are movable.

However, in the structures shown in FIG. 4, FIG. 5, and FIG. 6 and theirmodifications, in the display apparatus 1 of this embodiment, an imagesignal processed in the display image processing section 12 and suppliedto the display driving section 13 becomes a display image signal that isfinally displayed on the display sections 2. In this example, especiallya display image signal is a signal that causes an image to be displayedas a scene difference from a scene that the user sees through thedisplay sections 2 that is caused to be in the through-state.

An image as a display image signal, namely, as a scene different from ascene that user sees through the display sections 2 that is caused to bein the through-state can be contemplated to be obtained in variousmanners.

For example, images captured by the image capturing section 3 (3X, 3Y)and an image received by the communication section 26 may be images asscenes different from a scene that the user sees through the displaysections 2 or when an image as a scene that the user sees through thedisplay sections 2 is processed in the captured image signal processingsection 15 and the display image processing section 12 in predeterminedmanners, the resultant image may become an image as a scene differentfrom a scene that he or she sees through the display sections 2.

Specific examples of images of scenes different from a scene that theuser sees through the display sections 2 will be described later.

This example is based on such that when such an image is displayed,while a part of the screen area of each of the display sections 2 iscaused to be in the through-state, an image corresponding to an imagesignal that is output from the display image processing section 12 isdisplayed. In other words, an image is displayed on a part of an area ofeach of the display sections 2 caused to be in the through-state.

3. Areas on Screen

Examples in which a part of the screen area of each of the displaysections 2 is caused to be in the through-state and an image signal thatis output from the display image processing section 12 is displayed areshown in FIG. 7A, FIG. 7B, FIG. 7C, FIG. 7D, FIG. 7E, and FIG. 7F, andFIG. 8A, FIG. 8B, FIG. 8C, and FIG. 8D.

In the display sections 2, in a mode in which an image is displayedwhile a through-state area is left, a parent screen area and a childscreen area are set in the screen area, one of the parent screen areaand the child screen area is set to the through-state, and the other isset to an image display state with a display image signal.

Instead, the screen area may be divided into an area that is thethrough-state and an area in which an image corresponding to a displayimage signal is displayed.

In FIG. 7A, FIG. 7B, FIG. 7C, FIG. 7D, FIG. 7E, FIG. 7F, and FIG. 8A,FIG. 8B, FIG. 8C, and FIG. 8D, it is assumed that an area AR1 is an areain the through-state and an area AR2 is an area in which an imagecorresponding to a display image signal is displayed.

FIG. 7A shows an example in which the area AR2 in the screen area ofeach of the display sections 2 is set as a child screen at the lowerright of the area AR1 and an image is displayed on the area AR2.

FIG. 7B shows an example in which the area AR2 in the screen area ofeach of the display sections 2 is set as a child screen at the lowerleft of the area AR1 and an image is displayed on the area AR2.

When the child screen is set in such a manner, the area AR2 may be setas the child screen at various positions such as upper right, upperleft, center, rightward center, and leftward center (not shown).

FIG. 7C shows an example in which the size of the area AR2 is decreased.

FIG. 7D shows an example in which the size of the area AR2 is increased.

FIG. 7E shows an example in which the screen area of each of the displaysections 2 is equally divided into the area AR1 and the area AR2 as leftand right areas, respectively.

FIG. 7F shows an example in which the screen area of each of the displaysections 2 is equally divided into the area AR1 and the area AR2 asupper and lower areas, respectively.

For example, in the structures shown in FIG. 4 and FIG. 5, when an imageis displayed, while the area AR1 is caused to be in the through-state,it can be contemplated that the area AR2 is caused to display an imagein one of the modes shown in FIG. 7A, FIG. 7B, FIG. 7C, FIG. 7D, FIG.7E, and FIG. 7F.

Instead, the system controller 10 may select one of the area forms shownin FIG. 7A, FIG. 7B, FIG. 7C, FIG. 7D, FIG. 7E, and FIG. 7F to displayan image or change them based on information (for example, user'soperation information, information about the user, external information,and so forth) received from the operation trigger information generationsection 16. For example, the system controller 10 can change theposition of the area AR2 as the child screen as shown in FIG. 7A or FIG.7B, change the size of the area AR2 as shown in FIG. 7C or FIG. 7D, orequally divide the display area, for example, according to a user'soperation or the like such that the area AR2 is formed according to hisor her preference.

Instead, it can be contemplated that the area AR1 and the area AR2 shownin FIG. 7A, FIG. 7B, FIG. 7C, FIG. 7D, FIG. 7E, and FIG. 7F are causedto display an image and to be in the through-state, respectively, asso-called display area switching.

Although FIG. 8D shows an example in which the entire screen area is thearea AR2 in which an image corresponding to a display image signal isdisplayed, for example, the display state shown in FIG. 7A, FIG. 7B,FIG. 7C, FIG. 7D, FIG. 7E, or FIG. 7F may be switched to the displaystate in which an image is displayed without the through-state area asshown in FIG. 8D.

FIG. 8A shows an example in which two child areas AR2a and AR2b are setat the lower right and lower left of the area AR1, respectively, of thescreen area of each of the display sections 2 and an image is displayedon the child areas AR2a and AR2b.

FIG. 8B shows an example in which areas AR2a and AR2b are set at theleft and right of the area AR1, respectively, in the screen area of eachof the display sections 2 and an image is displayed on the areas AR2aand AR2b.

FIG. 8C shows an example in which areas AR2a and AR2b are set above andbelow the area AR1, respectively, in the screen area of each of thedisplay sections 2 and an image is displayed on the areas AR2a and AR2b.

When there are two image capturing functional blocks as shown in FIG. 6,respective images corresponding to captured images of the individualimage capturing functional blocks may be displayed on the two areas AR2aand AR2b.

Instead, the areas AR1, AR2a, and AR2b may be equally divided into threeareas.

In addition, it can be supposed that an area form is selected or areasare switched under the control of the system controller 10 like the caseshown in FIG. 7A, FIG. 7B, FIG. 7C, FIG. 7D, FIG. 7E, and FIG. 7F.

In addition, the state shown in FIG. 8A, FIG. 8B, or FIG. 8C may beswitched to the state in which an image is displayed on the entirescreen as shown in FIG. 8D.

Even if there are a plurality of display image sources of the pluralityof image capturing functional blocks and the communication section 26,it can be contemplated that one of them is selected and an image thereofis displayed in the area forms as shown in FIG. 7A, FIG. 7B, FIG. 7C,FIG. 7D, FIG. 7E, and FIG. 7F.

When the display apparatus 1 is structured to have three or more displayimage sources, three or more areas AR2 may be set and images from therespective display image sources may be displayed thereon at the sametime.

When the display sections 2 are used for the right eye and the left eyeof the user as shown in FIG. 1A, it can be contemplated that the entirescreen of one display section 2 is the through-state and that of theother display section 2 displays an image as shown in FIG. 8D.

4. Exemplary Display Operation

In the display apparatus 1 having the structures shown in FIG. 4, FIG.5, and FIG. 6, an image signal processed in the display image processingsection 12 and supplied to the display driving section 13 is a displayimage signal that is finally displayed on the display sections 2. Inparticular, this display image signal is displayed on a part of an areaon the screen while the through-state area is left as described withreference to FIG. 7A, FIG. 7B, FIG. 7C, FIG. 7D, FIG. 7E, FIG. 7F, andFIG. 8A, FIG. 8B, FIG. 8C, and FIG. 8D. In addition, a display imagesignal is a signal of an image of a scene different from a scene(namely, a scene that the user sees in the through-state) through thedisplay sections 2.

Although an image as a display image signal is an image captured by theimage capturing section 3 (or 3X, 3Y) or an image received by thecommunication section 26, display image signals from sources of thesesections are images of scenes different from a scene that the user seesin the through-state and while the user is seeing an external scenethrough the through-state area, he or she sees a display image through apart of an area such that the user's visual sense is extended.

Next, various exemplary operations under the control of the systemcontroller 10 will be described.

FIG. 9 shows a control process of the system controller 10.

Step F101 shows a control process in which the system controller 10controls the display control section 14 to cause the display sections 2to be in the through-state. For example, in the initial phase when thedisplay apparatus 1 is turned on, the system controller 10 controls thedisplay sections 2 to be in the through-state at step F101.

While the entire screen of each of the display sections 2 is caused tobe in the through-state, the system controller 10 determines whether ornot a display start trigger has occurred at step F102. For example, byproviding an operation switch that the user can operate as the operationtrigger information generation section 16 and causing the user tooperate a predetermined switch, it can be determined that a trigger thatstarts the display state have occurred. Instead, as will be exemplifiedlater, by causing the operation trigger information generation section16 to detect the user's situation and external situation, the systemcontroller 10 may determine that a display start trigger has occurredcorresponding to a predetermined condition.

When the determined result denotes that a display start trigger haveoccurred, the system controller 10 causes the process to advance to stepF103 and executes the display start control for the display imagesignal. In other words, the system controller 10 controls the displaysections 2 to display an image signal captured by the image capturingsection 3 (or 3X, 3Y) or a display image signal based on an image signalreceived by the communication section 26. In addition, at this point,for example, as described in FIG. 7A, FIG. 7B, FIG. 7C, FIG. 7D, FIG.7E, FIG. 7F, and FIG. 8A, FIG. 8B, FIG. 8C, and FIG. 8D, an image isdisplayed on a part of an area of the screen.

After the image is displayed, namely, while a part of the screen of eachof the display sections 2 is caused to be through-state area and anotherpart is an area on which an image is displayed, the system controller 10monitors whether or not a display switching trigger has occurred at stepF104 and whether or not a display end trigger has occurred at step S105.

Occurrence of the display switching trigger at step F104 means that thesystem controller 10 has determined that display images be switchedbased on the user's operation, user's situation, or external situation.

Switching of display images includes, for example, switching of displayimages and switching of area.

It can be contemplated that switching of display images includesremarkably-various examples, for example, changing of images by a zoomprocess and changes of focus positions in the image capturing section 3,changes of images by changes of image capturing sensitivities, changesof images by signal processes in the captured image signal processingsection 15, changes of images by changes of image capturing directions(directions of objects) when the image capturing section 3 is a movablecamera, changes of images by signal processes of the display imageprocessing section 12, and changes of images by switching of sourceswhen there are a plurality of sources of display image signals.

The switching of area forms includes changes of positions of parent andchild screens, switching of parent and child screens, changes ofpositions and switching of areas of divided screens, switching to a fullscreen display, and so forth and is controlling, for example, switchingfrom the state shown in FIG. 7A to the state shown in FIG. 7B, switchingfrom the state shown in FIG. 7A to the state of FIG. 7E, or switchingfrom the state shown in FIG. 7A to the state shown in FIG. 8D.

With respect to the display end trigger at step F105, for example, whenthe user performs a display end operation with a predetermined operationswitch, the system controller 10 can determine that a display state endtrigger has occurred; instead, the user's situation or externalsituation is detected, and the system controller 10 may determine that adisplay end trigger has occurred according to a predetermined condition.

When the determined result denotes that an image control trigger hasoccurred, the system controller 10 causes the process to advance fromstep F104 to step F106 and controls switching for image displayoperations. Thus, images displayed on a part of an area or area forms ofeach of the display sections 2 are switched.

After the system controller 10 has controlled display switching at stepF106, the system controller 10 monitors whether or not a trigger hasoccurred at steps F104 and F105.

When the determined result denotes that a display end trigger hasoccurred, the system controller 10 returns the process from step F105 tostep F101 and causes the display control section 14 to end the imagedisplay and cause the entire surface of each of the display sections 2to be in the through-state.

While the user is wearing the display apparatus 1 and its power is beingturned on, the system controller 10 performs a control process, forexample, as shown in FIG. 9.

In this process, an image is displayed at step F103 and displayswitching is performed at step F106 and exemplary images displayed inthese processes will be described with reference to FIG. 10A and FIG.10B, FIG. 11A and FIG. 11B, FIG. 12A and FIG. 12B, FIG. 13A and FIG.13B, FIG. 14A and FIG. 14B, and FIG. 15A, FIG. 15B, FIG. 15C, and FIG.15D.

In FIG. 10A and FIG. 10B, FIG. 11A and FIG. 11B, FIG. 12A and FIG. 12B,FIG. 13A and FIG. 13B, and FIG. 14A and FIG. 14B, it is assumed thatbasically with the exemplary structure shown in FIG. 4, the imagecapturing section 3 is structured to capture an image of a scene infront of the user, namely, in the same direction as a scene that he orshe sees through the through-state area.

It is assumed that FIG. 10A shows the case in which the entire surfaceof each of the display sections 2 is the through-state. In other words,the display sections 2 are simply transparent plate members and the usersees a visual scene through the transparent display sections 2.

FIG. 10B shows an image display state in which an image captured, forexample, by the image capturing section 3 is displayed on the displaysections 2. In this example, the area AR1 is caused to be in thethrough-state, whereas the area AR2 displays an image. In this case,assuming that the image capturing section 3 captures an image in frontof the user, when the image capturing section 3 is zoom-controlled tocapture a telescopic image, an image (telescopic image) of a scene thatis different from the visual field of the user through the through-statearea AR1 is displayed on the area AR2. Thus, while seeing an ordinaryscene, the user can enjoy a telescopic image that he or she cannot seewith his or her ordinary visual sense.

In this example, although a telescopic image is obtained by theoperation of the image capturing section 3, when it is zoomed with awide angle, an image of a near scene is displayed with a wide angle onthe display sections 2. The telescopic, wide angle controls can beperformed by a signal process of the display apparatus 15 as well as thezoom-lens drive control of the image capturing section 3.

Instead of the so-called zoom operation, the system controller 10 maycause the image capturing section 3 to change its focus distance so asto capture an image of a near scene or far scene and display thecaptured image on the area AR2 of each of the display sections 2 (notshown).

FIG. 11A shows the case in which the entire surface of each of thedisplay sections 2 is caused to be in the through-state.

FIG. 11B shows an image display state in which the system controller 10causes the display control section 14 (display image processing section12 and display driving section 13) to perform an enlargement process foran image obtained from the image capturing section 3 such that the areaAR1 is caused to be in the through-state and the area AR2 displays anenlarged image as shown in FIG. 11B. Since an image of a scene that isdifferent from the user's visual field through the through-area AR1 isdisplayed as an enlarged image on the area AR2, he or she can see animage that he or she cannot see with his or her ordinary visual sensewhile seeing an ordinary scene.

In contrast, it can be contemplated that the system controller 10 causesthe lighting section 4 (display image processing section 12 and displaydriving section 13) to execute a reduction process for an image obtainedfrom the image capturing section 3 and display a reduced image on thearea AR2.

FIG. 12A shows the case in which the entire surface of each of thedisplay sections 2 is caused to be in the through-state and especiallythe environment is dark.

FIG. 12B shows the state in which an image is displayed in the case thatthe environment is dark and that the system controller 10 causes theimage capturing control section 11 (image capturing section 3 and thecaptured image signal processing section 15) to increase the imagecapturing sensitivity and the captured image signal processing section15 and the display image processing section 12 to adjust the luminancelevel, contrast, and sharpness such that a more clear and bright displayimage signal is obtained and displayed. In other words, in this example,the area AR1 is caused to be in the through-state and an image whoseluminance, namely brightness, is increased is displayed on the area AR2.Since an image of a scene different from the user's visual sight throughthe through-state area AR1 is displayed as a brightness-adjusted image,the user can see an image that he or she cannot see with his or herordinary visual sense.

Instead, when the lighting section 4 is caused to perform the lightingoperation, such an image can be preferably captured.

In contrast, when the environment is too bright, it can be contemplatedthat the system controller 10 causes the image capturing control section11 (image capturing section 3 and captured image signal processingsection 15) to decrease the image capturing sensitivity and the capturedimage signal processing section 15 and the display image processingsection 12 to adjust luminance level, contrast, and sharpness such thata non-glaring display image signal is obtained and displayed.

FIG. 13A shows the state in which although the entire surface of each ofthe display sections 2 is caused to be in the through-state, forexample, the user is staying in a dark bed room where a child issleeping, the user cannot clearly see anything nearly in the dirk.

FIG. 13B shows the state in which an image is displayed. When the systemcontroller 10 causes the image capturing control section 11 (imagecapturing section 3 and captured image signal processing section 15) toincrease the infrared image capturing sensitivity, an image whoseinfrared image capturing sensitivity is increased is displayed on thearea AR2. In other words, an image is displayed such that the user cansee the child's sleeping face and so forth. Thus, the user can see anight vision image that he or she cannot see with his or her ordinaryvisual sense.

FIG. 14A shows the case in which the entire surface of each of thedisplay sections 2 is caused to be in the through-state.

FIG. 14B shows the state in which although an image is displayed, whenthe system controller 10 causes the image capturing control section 11(image capturing section 3 and captured image signal processing section15) to increase the ultraviolet image capturing sensitivity, an imagewhose ultraviolet image capturing sensitivity is increased is displayedon the area AR2. Thus, the user can see an image of an ultravioletcomponent that he or she cannot see with his or her ordinary visualsense.

Although FIG. 10A and FIG. 10B; FIG. 11A and FIG. 11B; FIG. 12A and FIG.12B; FIG. 13A and FIG. 13B; and FIG. 14A and FIG. 14B show examples inwhich the image capturing section 3 structured shown in FIG. 4 isdisposed such that it captures an image in front of the user, FIG. 15A,FIG. 15B, FIG. 15C, and FIG. 15D show examples in which the imagecapturing section 3 shown in FIG. 4 (or image capturing sections 3X and3Y shown in FIG. 6) is disposed such that it captures an image in adirection different from the forward direction that the user sees.

FIG. 15A shows the case in which the entire surface of each of thedisplay sections 2 is caused to be in the through-state.

If the image capturing section 3 is disposed such that it captures animage in the rear of the user, the image is displayed as shown in FIG.15B where the area AR1 is caused to be in the through-state and theimage in the rear of the user is displayed on the area AR2.

If the image capturing section 3 is disposed such that it captures animage in the upper direction of the user, the image is displayed asshown in FIG. 15C where the area AR1 is caused to be in thethrough-state and the image captured in the upper direction of the useris displayed on the area AR2.

In these cases, since an image of a scene different from the user'svisual field through the through-state area AR1 is displayed on the areaAR2, while he or she is seeing a scene in his or her forward direction,he or she can see an image in a direction that he or she cannot see withhis or her ordinary visual sense (backward image and upward image).

FIG. 15D shows an exemplary image display in which the image capturingsections 3X and 3Y structured to have the plurality of image capturingfunctional blocks shown in FIG. 6 are disposed such that they captureimages in the user's right backward direction and left backwarddirection, respectively. In other words, the area AR1 is caused to be inthe through-state and images captured in the user's right backwarddirection and left backward direction are displayed on the areas AR2aand AR2B, respectively.

In this case, since images of scenes different from the user's visualfield through the through-state area AR1 are displayed on the areas AR2aand AR2b and the user can see scenes in his or her right backwarddirection and left backward direction in addition to the ordinaryforward scene on the through-state area AR1, he or she can check fornearly all circumference.

Although various display examples have been described, they are justexemplary. In this embodiment, by controlling the processes andoperations of the image capturing section 3 (3X, 3Y), captured imagesignal processing section 15, display image processing section 12,display driving section 13, and communication section 26, variousdisplay forms are accomplished.

Next, exemplary image signals displayed along with the through-statearea on the screen, namely exemplary display image signals that causeimages to be displayed as scenes different from a scene that the usersees through the display sections 2 caused to be in the through-state,will be described.

First, when the image capturing section 3 structured as shown in FIG. 4is disposed such that it captures an image in the forward direction ofthe user (namely, the image capturing section 3 captures an image of ascene that the user can see in the through-state), examples of displayimage signals based on a captured image signal obtained in the imagecapturing section 3 are as follows:

a display image signal as a telescopic zoom image obtained by the zoomoperation of the image capturing section 3,

a display image signal as a wide zoom image obtained by the zoomoperation of the image capturing section 3,

a display image signal as a distant view image obtained by the focuscontrol operation of the image capturing section 3,

a display image signal as a near view image obtained by the focuscontrol operation of the image capturing section 3,

a display image signals as a high speed captured image and a low speedcaptured image obtained by changing capturing frame rates of the imagecapturing section 3,

a display image signal for which the enlargement process has beenperformed as a signal process of the captured image signal processingsection 15 or the display image processing section 12,

a display image signal for which the reduction process has beenperformed as a signal process of the captured image signal processingsection 15 or the display image processing section 12,

a display image signal as an image whose brightness has beenstrengthened by increasing the image capturing sensitivity of the imagecapturing section 3,

a display image signal as an image whose brightness has been suppressedby decreasing the image capturing sensitivity of the image capturingsection 3,

a display image signal as a night vision image obtained by increasingthe infrared image capturing sensitivity of the image capturing section3,

a display image signal as an image obtained by increasing theultraviolet image capturing sensitivity of the image capturing section3,

a display image signal as an image captured by the image capturingsection 3 such that a particular wavelength band is cut,

a display image signal as an image whose brightness has beenstrengthened by increasing the luminance in a signal process of thecaptured image signal processing section 15 or the display imageprocessing section 12,

a display image signal as an image whose brightness has been suppressedby decreasing the luminance in a signal process of the captured imagesignal processing section 15 or the display image processing section 12,

a display image signal as an image whose hue and atmosphere have beenchanged in the color signal process of the captured image signalprocessing section 15 or the display image processing section 12,

a display image signal as an image whose contrast, sharpness, and soforth have been adjusted in the signal process of the captured imagesignal processing section 15 or the display image processing section 12,

a display image signal as an image for which an image effect processsuch as mosaic/luminance inversion/soft focus/partial emphasis ofimage/highlight display has been performed as a signal process of thecaptured image signal processing section 15 or the display imageprocessing section 12, and

display image signals obtained in combinations of the foregoingindividual operations and signal processes.

When the image capturing section 3 captures an image of a scene in frontof the user who can see in the through-state, for example, by obtaininga display image signal in such an operation or process, the displayimage signal becomes a signal of an image of a scene different from ascene that he or she sees through the display sections 2 caused to be inthe through-state.

Of course, it can be contemplated that there are many types of displayimage signals as “scenes different from the through-state” obtained bythe operation of the image capturing section 3, the signal process ofthe captured image signal processing section 15, and the signal processof the display image processing section 12.

In addition, it can be contemplated that for example the magnificationof the zoom process and enlargement/reduction process, levels in imagecapturing sensitivity, process coefficients of luminance signal processand color signal process, and so forth are changed as desired.

When the image capturing section 3 is disposed such that it captures animage in a direction different from the forward direction of the user,exemplary display image signal based on captured image signals obtainedby the image capturing section 3 may be contemplated as follows.

In other words, in this case, since a captured image ordinarily obtainedby the image capturing section 3 is a captured image of a scenedifferent from a scene that the user ordinarily sees through the displaysections 2 caused to be in the through-state (for example, backwardscene, upward scene, downward scene, rightward scene, leftward scene),the captured image signal can be directly displayed as a display imagesignal on the display sections 2.

In addition, in this case, as described above, an image signal for whichthe image capturing operation of the image capturing section 3, thesignal process of the captured image signal processing section 15, andthe signal process of the display image processing section 12 have beenperformed may be a display image signal.

In addition, these processes can be applied to the case in which aplurality of image capturing functional blocks are provided as shown inFIG. 6. If there is an image capturing section 3 that captures an imagein the forward direction in the plurality of image capturing functionalblocks, the captured image signal of the image capturing section 3 maybe a display image signal of an image signal of a different scene forwhich the image capturing operation of the image capturing section 3,the signal process of the captured image signal processing section 15,and the signal process of the display image processing section 12 havebeen performed.

When there is an image capturing section 3 that captures an image in adirection other than the forward direction, the captured image signalcan be used as a display image signal or an image signal for which theimage capturing operation of the image capturing section 3, the signalprocess of the captured image signal processing section 15, and thesignal process of the display image processing section 12 have beenperformed can be used as a display image signal.

When there is a movable image capturing section that can change theimage capturing direction, likewise, a captured image other than in theforward direction of the user or an image signal for which the imagecapturing operation of the image capturing section 3, the signal processof the captured image signal processing section 15, and the signalprocess of the display image processing section 12 have been performedcan be used as a display image signal.

In the movable type, moving control that tracks a particular object canbe contemplated. For example, an image of a captured image signal isanalyzed and when a particular object is detected, the image capturingdirection is changed corresponding to the motion of the particularobject in the captured image. Such control allows the user to see animage that tracks a particular object, for example, on the area AR2.

It can be contemplated that such processes can be applied to the case inwhich a captured image of an external image capturing device 70 receivedby the communication section 26 is displayed.

In other words, since a captured image of the external image capturingdevice 70 is a captured image of a scene that is different from a scenethat the user ordinarily sees through the display sections 2 caused tobe in the through-state, by directly displaying an image signal receivedby the communication section 26 as a display signal on the displaysections 2, the user is provided with a variety of images.

Of course, in such a case, image signals for which the foregoing signalprocesses of the display image processing section 12 have been performedmay be display image signals.

For example, when the user is watching a football game at a stadium seatas shown in FIG. 10A, it can be contemplated that an image that theimage capturing device 70 captures at another place of the stadium isreceived by the communication section 26 and displayed on the area AR2of each of the display sections 2 as shown in FIG. 10B; when images arereceived from an image capturing device 70 installed, for example, neara coach seat or an image is received from a small image capturing device70 that an umpire wears, by displaying the received image on the displaysections 2, the user can watch the game with more interest.

Instead, many examples such as an example in which an image captured byan image capturing device 70 installed at a resort area or an imagecaptured by an image capturing device 70 to which an acquaintance who istravelling belongs is received by the communication section 26 anddisplayed on the area AR2 of the display sections 2 and an example inwhich a ground image (bird view image) captured by an image capturingdevice 70 with which an airplane or a satellite is equipped is receivedby the communication section 26 and displayed on the display area AR2 ofeach of the display sections 2 can be contemplated and with such adisplay, the user can enjoy a scene that he or she cannot ordinarilysee.

5. Exemplary Operation Triggers

As described in the process shown in FIG. 9, when the system controller10 determines that a display start trigger have occurred at step F102 ora display switching trigger have occurred at step F104, the systemcontroller 10 executes displaying. When the system controller 10determines that a display end trigger have occurred at step F105, thesystem controller 10 ends displaying.

Although triggers with respect to these display operations can occurcorresponding to user's operations, the system controller 10 detects auser's situation and an external situation and determines whether or nota trigger has occurred corresponding to a predetermined condition asdescribed above.

Here, examples in which it is determined whether or not a trigger hasoccurred based on a user's situation and an external situation will bedescribed.

When it is determined whether or not a trigger has occurredcorresponding to a user's situation, the display apparatus 1 is providedwith as the operation trigger information generation section 16 a visualsensor, an acceleration sensor, a gyro, a biological sensor, and soforth.

Although the visual sensor detects information about a user's visualsense, the visual sensor can be composed of an image capturing sectionthat is disposed near one of the display sections 2 and is caused tocapture an image of his or her eye. By causing the system controller 10to fetch and analyze the image of the eye of the user captured by theimage capturing section, a visual line direction, a focus distance, adilation of pupils, an eye fundus pattern, eyelid opening/closing, andso forth can be detected and thereby his or her situation and intentioncan be determined based on the detected results.

Instead, the visual sensor may be composed of a light emitting sectionthat is disposed near one of the display sections 2 and that emits lightto the eye of the user and a light receiving section that receives lightreflected from the eye. By detecting the thickness of the lens of theuser's eye, for example, with a signal corresponding to the receivedlight, the focus distance of the user's eye can be detected.

By detecting the visual line direction of the user, the systemcontroller 10 can determine a portion on which the user is focusing, forexample, in an image displayed on the display sections 2.

In addition, the system controller 10 can recognize the visual linedirection of the user as an operation input. For example, when the usermoves the visual line to the left and right, the system controller 10can recognize these operations as predetermined operation inputs to thedisplay apparatus 1.

By detecting the focus distance of the user, the system controller 10can determine whether a scene on which the user is focusing is far ornear and perform zoom control, enlargement control, reduction control,and so forth corresponding to the determined result. For example, whenthe user sees a far scene, the system controller 10 may perform thetelescopic display operation.

When the dilation of the user's pupil is detected in the through-state,the brightness of the surroundings can be determined; when the dilationof the user's pupil is detected in the monitor display state, the glareand so forth that the user feel for an image that is displayed can bedetermined and the luminance, the image capturing sensitivity, and soforth can be adjusted corresponding to the determined results.

Detection of the eye fundus pattern of the user can be used for personalauthentication of the user. Since the eye fundus pattern is unique foreach user, he or she who wears the display apparatus 1 can be identifiedcorresponding to the eye fundus pattern and control can be performedcorresponding to him or her or control can be performed such that adisplay operation is performed for only a particular user.

When the eyelid opening/closing operations of the user are detected, theglare and eyestrain that he or she feels can be determined. In addition,the opening/closing operations of the eyelids can be recognized asuser's intentional operation inputs. When the user has performed theeyelid opening/closing operations three times, these operations may bedetermined as a predetermined operation input.

The acceleration sensor and the gyro output signals corresponding to themotion of the user. For example, it is suitable that the accelerationsensor detects a motion in the linear direction and the gyro detects amotion and a vibration of a rotating system.

The acceleration sensor and the gyro can detect a motion of the whole ora part of the body of the user although depending on the positions wherethey are disposed.

When the acceleration sensor and the gyro are disposed in theeyeglass-type display apparatus 1 shown in FIG. 1A and FIG. 1B, namelywhen the acceleration sensor and the gyro detect a motion of the head ofthe user, information of the acceleration sensor is accelerationinformation as the motion of the user's head or his or her whole body,whereas the information of the gyro is information of an angularvelocity and a vibration as a motion of the user's head or his or herwhole body.

Thus, an action of which the user moves the head from the neck can bedetected. For example, the state in which the user is looking up and thestate in which he or she is looking down can be determined. When theuser is looking down, it can be determined that he or she is seeing anear object, for example, he or she is reading a book or the like. Incontrast, when the user is looking up, it can be determined that he orshe is seeing a far object.

When the system controller 10 has detected a user's moving action of thehead from the neck, the system controller 10 may recognize it as auser's intentional action. For example, if the user has shaken his orher neck to the left twice, the system controller 10 can determine theaction as a predetermined operation input.

The acceleration sensor and the gyro can determine whether the user isin a stop state (non-walking state), a walking state, or a runningstate. In addition, the acceleration sensor and the gyro can detectchanges of states from a standing state to a sitting state or viceversa.

When the acceleration sensor and the gyro are separated from theheadgear-type mounting unit and mounted on an arm or a foot the user,they can detect the motion of only the arm or foot.

Although the biological sensor detects as user's biological information,for example, heart rate information (heart rate), pulse rate information(pulse rate), perspiration information, cerebral wave information(information of for example α wave, β wave, θ wave, and δ wave),galvanic skin response, body temperature, blood pressure, respiratoryactivity (for example, speed and depth of breathing, amount ofventilation, etc.), and so forth of the user, the system controller 10can determine whether he or she is a tense state, an excited state, acalm state, a drowsy state, a comfortable state, an uncomfortable state,or the like corresponding to these kinds of information.

In addition, it can be determined that the user have worn the displayapparatus 1 corresponding to biological information. For example, whenthe user is not wearing the display apparatus 1, the system controller10 may control the display apparatus 1 to operate in a standby state inwhich only biological information is detected. When the systemcontroller 10 has detected that the user has worn the display apparatus1 corresponding to the detected biological information, the systemcontroller 10 may turn on the power of the display apparatus 1. Incontrast, when the user has taken off the display apparatus 1, thesystem controller 10 may control the display apparatus 1 to be restoredto the standby state.

In addition, information detected by the biological sensor can be usedto personally authenticate the user (identify the user who has worn thedisplay apparatus 1).

When the biological sensor is disposed in the mounting frame of theeyeglass-type display apparatus 1, the foregoing information may bedetected at the bilateral temporal region or the occipital region of theuser or the biological sensor may be separated from the mounting frameof the display apparatus 1 and disposed at a predetermined position ofthe user's body.

When an external situation is detected, as a structure that obtainsexternal information, the display apparatus 1 is provided withsurrounding environment sensors, an image capturing target sensor, a GPSreceiving section, a date and time counting section, and an imageanalyzing section in the operation trigger information generationsection 16 or the communication section 26 is used.

The surrounding environment sensors are supposed to be a luminancesensor, a temperature sensor, a humidity sensor, an atmospheric pressuresensor, and so forth.

The luminance sensor may detect information about the surroundingbrightness of the display apparatus 1.

The temperature sensor, humidity sensor, and atmospheric pressure sensorcan obtain information with which the temperature, humidity, atmosphericpressure, and weather can be determined depending on their types.

Since these surrounding environment sensors can determine thesurrounding brightness, outdoor weather situation, and so forth, thesystem controller 10 can execute and control a generation operation of adisplay image signal suitable for the surrounding brightness and weathersituation using these information as the external information. Forexample, the luminance level is increased or decreased corresponding tothe brightness of the surroundings while the atmosphere of an image ischanged corresponding to the weather condition.

The image capturing target sensor detects information about the imagecapturing target. Although the image capturing target sensor can becontemplated to be a distance measurement sensor or a pyroelectricsensor, the distance to an image capturing target and information withwhich the image capturing target is determined can be obtained.

By detecting the distance to the image capturing target, the systemcontroller 10 can execute and control the image capturing operation andthe display operation corresponding to the detected distance. Bydetecting that the image capturing target is a living body such as ahuman, the system controller 10 can execute and control a generationoperation of a display image signal corresponding to the image capturingtarget.

The GPS receiving section obtains information of latitude and longitudeof the current position. When the latitude and longitude of the currentposition have been detected, with reference to a map database or thelike, information about the current position (information about theneighborhood of the current position) can be obtained. When a recordmedium (such as a HDD (Hard Disk Drive) or a flash memory) that has arelatively large recording capacity is mounted as a record medium towhich the system controller 10 can refer and the record medium hasstored a map database, information about the current position can beobtained.

Even if the display apparatus 1 is not provided with a built-in mapdatabase, when the display apparatus 1 is structured to have thecommunication section 26, the display apparatus 1 may cause thecommunication section 26 to access for example a network server or adevice having a built-in map database, transmit information of latitudeand longitude of the current position to the network server or thedevice, request the network server or the device to transmit informationabout the current position to the communication section 26, and receivethe information.

Information associated with the current position includes place names,building names, facility names, store names, station names, and so forthin the vicinity of the current position.

In addition, information associated with the current position includesinformation representing the types of structures such as parks, themeparks, concert halls, theaters, movie theaters, sports facilities, andso forth.

In addition, information associated with the current position includestypes and names of natural things such as seashores, ocean surfaces,rivers, mountains, mountain tops, forests, lakes, plains, and so forth.

As information about more detailed positions, areas in theme parks, seatareas of baseball stadiums, soccer stadiums, etc., seat areas of concerthalls, and so forth can be obtained as information about the currentposition.

By obtaining information about the current position, the systemcontroller 10 can execute and control a generation operation of adisplay image signal and perform display start control or display endcontrol at particular positions corresponding to the current position,geographic conditions, facilities, and so forth in the vicinity of thecurrent position.

The date and time counting section counts, for example, the years,months, days, hours, minutes, and seconds. The system controller 10 canrecognize the current time, day or night, month, season, and so forthcorresponding to the counted results of the date and time countingsection. Thus, the system controller 10 can execute and control ageneration operation for a display image signal corresponding, forexample, to day or night (time) and that corresponding to the currentseason.

When an image analyzing section that analyzes a captured image isprovided, the following various types of information about an imagecapturing target can be detected from a captured image.

First, a person, an animal, a natural thing, a building, a machine, andso forth can be identified as types of image capturing targets from acaptured image. As animals, a situation in which a bird has beencaptured as an object, a situation in which a cat has been captured asan object, and so forth can be identified. As natural things, sea,mountains, trees, rivers, lakes, sky, sun, moon, and so forth can beidentified. As structures, houses, buildings, stadiums, and so forth canbe identified. As devices, situations in which a personal computer, anAV (Audio Visual) device, a mobile phone, a PDA, an IC card, atwo-dimensional bar code, and so forth are image capturing targets canbe identified.

When characteristics of the shapes of various types of image capturingtargets are pre-registered, it can be determined whether or not anobject corresponding to the registered characteristics is contained in acaptured image.

In addition, the image analyzing section can analyze an image to detecta motion of an object, for example, a quick motion in an image, bydetecting the difference of adjacent frames of images. For example, theimage analyzing section can detect a situation in which an image of aquickly moving object is being captured, for example, while an image ofa player in a sports match or an image of a running car is beingcaptured.

In addition, the image analyzing section can analyze an image todetermine a surrounding situation. For example, the image analyzingsection can determine the brightness due to day, night, or weather andalso can recognize the strength of rain and so forth.

In addition, when a person is an object, the image analyzing section cananalyze an image to identify the person from the face. As well known, aperson's face can be registered as personal characteristic data that arerelative position information of structural elements of the face. Forexample, the ratio (Ed/EN) of the distance EN between the center of theeyes and the nose and the distance Ed of the eyes and the ratio (Ed/EM)of the distance EM between the center of the eyes and the mouth and thedistance Ed of the eyes are unique to each person and are not affectedby changes in appearance with a hair style and wearing things such aseyeglasses and so forth. In addition, it is known that they do notchange with aging.

Thus, when a captured image contains a person's face, the imageanalyzing section can analyze the image to detect the foregoing personalcharacteristic data.

When personal characteristic data have been detected from a capturedimage, if for example a HDD (Hard Disk Drive), a flash memory, or thelike has been provided as a record medium to which the system controller10 can refer and that has stored a persons database, personalinformation of the object can be obtained from the persons database.Even if the display apparatus 1 does not have a built-in personsdatabase, the system controller 10 may cause the communication section26 to access for example a network server or a device having a built-inpersons database, transmit personal characteristic data so as to requestthe server or device to transmit information to the communicationsection 26, and receive particular personal information therefrom.

If the user has registered personal information such as the name,organization, and so forth of each person whom the user had met to thepersons database along with personal characteristic data, when the usermeets a particular person (his or her image has been captured), thesystem controller 10 can retrieve information about the person from thepersons database.

If a persons database that has registered information about famouspeople and personal characteristic data has been prepared, when the usermeets a famous person, the system controller 10 can retrieve informationabout the person from the persons database.

Based on the detection information of the image analysis section, thesystem controller 10 can execute and control a generation process for adisplay image signal corresponding to a capturing object. For example,when an image of a particular object or a particular person is captured,it can be contemplated that a display image signal that highlights theparticular object is generated.

The communication section 26 can obtain various types of information asexternal information.

For example, as described above, the communication section 26 can obtaininformation retrieved by an external device corresponding to informationof latitude and longitude, personal characteristic data, and so forthtransmitted from the display apparatus 1.

In addition, the communication section 26 can obtain meteorologicalinformation such as weather information, temperature information, andhumidity information from an external device.

In addition, facility use information, photographyprohibition/permission information, facility guide information, and soforth can be obtained from an external device.

In addition, identification information of an external device itself canbe obtained. Examples of identification information include a devicetype, a device ID, and so forth of a device identified as a networkdevice in a predetermined communication protocol.

In addition, image data that are stored in an external device, imagedata that are being reproduced or displayed by an external device, andimage data that are being received by an external device can beobtained.

Based on information obtained by such a communication section 26, thesystem controller 10 can execute and control a generation process for adisplay image signal.

When the foregoing exemplified structural element is provided as theoperation trigger information generation section 16, the user'ssituation and external situation are detected, and display start/end anddisplay switching (changes of display contents and switching of areaforms), a display operation that is appropriate or interesting isaccomplished without necessity of a user's operation.

Other exemplary structures that detect user's situation and an externalsituation can be contemplated.

6. Effect of Embodiment, Exemplary Modification, and Exemplary Extension

Although embodiments of the present invention have been described,according to the embodiments of the present invention, by causing a partof the display area of each of the display sections 2 to be in thethrough-state and a display image signal to be displayed, while the usersees an ordinary scene in the through-state area, he or she can see animage of a scene different from a scene that he or she ordinarily sees.For example, while the user is seeing ahead through a through-statearea, he or she can see a telescopic image, an enlarged image, aspecially captured image, an image of a rear scene, or the like withdisplaying of a display image signal. Thus, a situation in which theuser sees a scene that he or she cannot see with his or her ordinaryvisual sense along with an ordinary scene takes place and thereby asituation in which the user's visual ability is pseudo-extended can becreated.

In addition, since at least a part of the screen area of each of thedisplay sections 2 is caused to be in the through-state, the user whowears the display apparatus 1 can live free of difficulties. Thus, in auser's ordinary life, the benefits of the display apparatus 1 can beeffectively obtained.

The appearance and structure of the display apparatus 1 are not limitedto the examples shown in FIG. 1A and FIG. 1B, FIG. 2A and FIG. 2B, FIG.4, FIG. 5, and FIG. 6. Instead, various exemplary modifications can becontemplated.

For example, a storage section that records an image signal captured bythe image capturing section 3 to a record medium may be provided.

In addition, an operation that transmits a captured image signal fromthe communication section 26 to another device may be performed.

If the storage section is provided, a reproduced image obtained from arecord medium of the storage section may be displayed as a display imagesignal on the display sections 2.

Data that are reproduced from a record medium are supposed to be allkinds of data to be recorded on the record medium and displayed such asmoving image content including movies and video clips, still imagecontent captured, for example, by a digital still camera and recorded onthe record medium, computer use data of electronic books and so forth,image data, text data, spreadsheet data, and so forth that the user hascreated with a personal computer and recorded on the record medium, gameimages based on game programs recorded on the record medium, and soforth.

It can be contemplated that the communication section 26 receives notonly an image signal from the external image capturing device 70, but animage provided by an external content source device (moving image/stillimage) and displays the image on the display sections 2.

The content source device can be, for example, an AV (audio-Visual)device such as a video device, a television tuner, a home server deviceor an information processing device such as a personal computer, a PDA(personal digital Assistant), or a mobile phone. Such a content sourcedevice can be contemplated to be one of a variety of devices to whichthe user themselves or its acquaintance belongs or a server device towhich a public or service company that provides various kinds of contentbelongs.

Data transmitted from the content resource device to the displayapparatus 1 are supposed to be all kinds of data to be displayed such asmoving image content including movies and video clips, still imagecontent captured, for example, by a digital still camera and recorded onthe record medium, data of electronic books and so forth, computer-usedata such as image data, text data, spreadsheet data, and so forth thatthe user has created with a personal computer and recorded on the recordmedium, game images, and so forth.

In addition, a microphone that collects a surrounding sound while animage is being captured and an earphone type speaker section thatoutputs a sound may be provided.

In addition, it can be contemplated that a character recognition sectionthat recognizes characters from an image and a sound synthesis sectionthat performs a sound synthesis process are provided such that ifcharacters are contained in a captured image, the sound synthesissection generates a sound signal of a reading sound and the speakersection outputs the sound signal.

The display image signal may be a still image. For example, a stillimage is captured at timing of a predetermined trigger and the capturedstill image is displayed on the display sections 2.

Although the display apparatus 1 is exemplified as an eyeglass-type orheadgear-type mounting unit, as long as the display apparatus of thepresent invention is structured to display an image in front of the eyesof the user, the display apparatus may be any type that the user wearswith any mounting unit such as a headphone type, a neckband type, an earhanging type, or the like. Instead, the display apparatus 1 may be aunit that is attached to eyeglasses, a visor, a headphone, or the likeusing a mounting member such as a clip.

As described above, according to the embodiments of the presentinvention, by causing a part of the display area of display means to bein the through-state and a display image signal to be displayed by thedisplay means, while the user can see an ordinary scene in thethrough-state area, he or she can see an image of a scene different froma scene that he or she ordinarily sees and thereby there is an effectthat a situation in which the user's visual ability is pseudo-extendedcan be created.

In addition, since at least a part of the screen area of the displaymeans is caused to be in the through-state, the user who wears thedisplay apparatus 1 can live free of difficulties. Thus, in a user'sordinary life, the benefits of the display apparatus of the presentinvention can be effectively obtained.

1. A display apparatus, comprising: a display device, disposed in frontof eyes of a user, and configured to display an image and cause a screenarea to be in a through-state which is transparent or semi-transparent;an image capturing device disposed such that the image capturing deviceis configured to perform capturing of a captured image in a directionwhich the user sees through the display device or in a directiondifferent from the direction which the user sees through the displaydevice; and processing circuitry configured to perform a signal processfor the captured image and generate a first displaying image, andreceive an image signal from an external device and generate a seconddisplaying image, and control a part of the screen area of the displaydevice in the through-state and execute displaying of one of the firstdisplaying image and the second displaying image.
 2. The displayapparatus as set forth in claim 1, wherein the processing circuitry isfurther configured to execute displaying of the first displaying imageand the second displaying image simultaneously.
 3. The display apparatusas set forth in claim 1, wherein the processing circuitry is furtherconfigured to set a parent screen area and a child screen area in thescreen area of the display device and cause one of the parent screenarea and the child screen area to be in the through-state and the otherto be in a screen display state.
 4. The display apparatus as set forthin claim 1, wherein the processing circuitry is further configured todivide the screen area of the display device into areas and cause one ofthe divided areas to be in the through-state and another of the dividedareas to be in an image display state.
 5. A method, comprising:displaying, by a display device disposed in front of eyes of a user, animage and causing a screen area to be in a through-state which istransparent or semi-transparent; capturing, by an image capturingdevice, a captured image in a direction which the user sees through thedisplay device or in a direction different from the direction which theuser sees through the display device; performing a signal process forthe captured image and generating a first displaying image, andreceiving an image signal from an external device and generating asecond displaying image; and controlling a part of the screen area ofthe display device in the through-state and displaying one of the firstdisplaying image and the second displaying image.
 6. The method as setforth in claim 5, further comprising: displaying the first displayingimage and the second displaying image simultaneously.
 7. The method asset forth in claim 5, further comprising: setting a parent screen areaand a child screen area in the screen area of the display device andcausing one of the parent screen area and the child screen area to be inthe through-state and the other to be in a screen display state.
 8. Themethod as set forth in claim 5, further comprising: dividing the screenarea of the display device into areas and causing one of the dividedareas to be in the through-state and another of the divided areas to bein an image display state.